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DETAILED ACTION 
Claim Suggestions 

1 . Examiner suggests that the invention point to exactly what is claimed as 
the invention, for example, at claim 9, the method of claim 8, wherein said 
performing a golden section search comprises: (1) initializing the another color 
matching parameter and a minimum color matching error... 

It is suggested that the applicant remove the word "the" when referring to 
"another color matching parameter". It will deter any misinterpretation of 
antecedent problems related to color matching parameters. 

Drawings 

2. Color photographs and color drawings are not accepted unless a petition 
filed under 37 CFR 1.84(a)(2) is granted. Any such petition must be 
accompanied by the appropriate fee set forth in 37 CFR 1.17(h), three sets of 
color drawings or color photographs, as appropriate, and, unless already 
present, an amendment to include the following language as the first 
paragraph of the brief description of the drawings section of the specification: 

The patent or application file contains at least one drawing executed in color. Copies of this 
patent or patent application publication with color drawing(s) will be provided by the Office 
upon request and payment of the necessary fee. 

Color photographs will be accepted if the conditions for accepting color 
drawings and black and white photographs have been satisfied. See 37 CFR 
1.84(b)(2). 
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Examiner notes that a petition for color photographs /drawings has not yet 
been filed as of the mailing of this office action. Appropriate correction is 
required. 

3. Drawings 3, 8-10 are objected to under 37 CFR 1.83(a) because they fail 

to show appropriate contrast and sufficient details to describe the claimed 

invention as laid out in the specification. For example, in Figure 3, there are 

not visible outliers shown in this Figure. It is important to disclose and exhibit 

♦ 

the claimed invention as intended with respect to the specification. Any 
structural detail that is essential for a proper understanding of the disclosed 
invention should be shown in the drawing. MPEP § 608.02(d). Corrected 
drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the 
Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the 
immediate prior version of the sheet, even if only one figure is being amended. 
The figure or figure number of an amended drawing should not be labeled as 
"amended." If a drawing figure is to be canceled, the appropriate figure must be 
removed from the replacement sheet, and where necessary, the remaining 
figures must be renumbered and appropriate changes made to the brief 
description of the several views of the drawings for consistency. Additional 
replacement sheets may be necessary to show the renumbering of the 
remaining figures. Each drawing sheet submitted after the filing date of an 
application must be labeled in the top margin as either "Replacement Sheet" or 
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"New Sheet" pursuant to 37 CFR 1.121(d). If the examiner does not accept the 
changes, the applicant will be notified and informed of any required corrective 
action in the next Office action. The objection to the drawings will not be held 
in abeyance. 

Claim Rejections - 35 USC §102 

4. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in 
this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a 
printed publication in this or a foreign country, before the invention thereof by the applicant for a 
patent. 

(b) the invention was patented or described in a printed publication in this or a foreign country or 
in public use or on sale in this country, more than one year prior to the date of application for 
patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent 
or (2) a patent granted on an application for patent by another filed in the United States 
before the invention by the applicant for patent, except that an international application 
filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application 
designated the United States and was published under Article 21(2) of such treaty in the 
English language. 



5. Claims 1-5, 16 and 17 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Masaki (US 7,215,812 Bl). 



Regarding Claim 1: Masaki discloses a method for color matching a first image 
and a second image, wherein a first region of the first image and a second 
region of the second image overlap ("An image processing method for carrying 
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out predetermined correction processing with respect to one or more items 
regarding the quality of color images... "at abstract), the method comprising: 
generating a first histogram of the first region; generating a second histogram 
of the second region (Refer to Figure 5); 

determining corresponding pixel values from the first and the second 
histograms (Refer to Figure 4, numeral #51); 

determining at least one parameter of an optoelectronic conversion function 
(OECF) that best matches the corresponding pixel values (Refer to Figure 4, 
numeral #41); 

and color matching the second image to the first image by applying the OECF 
with the at least one parameter to the second image (Refer to Figure 4, 
numerals #53-#59) 

Regarding Claim 2: Masaki discloses the method of claim 1, further 
comprising, prior to said generating a first histogram and generating a second 
histogram: removing a percentage of the overlapping pixels with the greatest 
difference in brightness (Refer to Figure 4, numeral #45). 

Regarding Claim 3: Masaki discloses the method of claim 1, wherein: said 
generating a first histogram comprises recording in a first plurality of pixel 
value bins a first plurality of numbers of pixels that have respective pixel 
values in the first region (Refer to Figures #41 and #43); and said generating a 
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second histogram comprises recording in a second plurality of pixel value bins 
a second plurality of numbers of pixels that have the respective pixel values in 
the second region (Refer to Figures #49 and #51). 

Regarding Claim 4: Masaki discloses the method of claim 3, wherein said 
determining corresponding pixel values in the first and the second histograms 
comprises generating a lookup table (LUT) storing a third plurality of numbers 
of pixels and their corresponding pixel values (Refer to Figure 1, numeral 202). 

Regarding Claim 5: Masaki discloses the method of claim 4, wherein said 
generating a lookup table comprises: 

^ (1 ) initializing all entries in the LUT to 0; 

(2) initializing a first loop by setting i ~ 0;/ - 0; reml - hl[0]; and rem2 = h2[0]; 

. (3) updating the LUT by setting min jrem = min(reml, rem2); rem! ~ reml - 
minjrem; and rem2 ~ rem! ~ mmjrem; and incrementing LUT[i][/] by minjrem\ 

(4) if reml ~ 0, then incrementing i and setting rem! = hl[0; . 

(5) if rem! = 0, then incrementing/ and setting reml = h2[/]; 

(6) if i < 256 andy < 256, then repeating steps (3) to (5); 

wherein h 1 [ ] is the number of pixels having a certain pixel value in the first 
histogram, h2[ ] is the number of pixels having a certain pixel value in the 
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second histogram, and LUT[ ][ ] is the number of pairs of corresponding pixel 
values having a certain pixel value in the first histogram and a certain pixel 
value in the second histogram (Refer to Figure 1, numeral 203). 

Regarding Claim 16: Masaki discloses a method for color matching a first 
image and a second image, wherein a first region of the first image and a 
second region of the second image overlap ("An image processing method for 
carrying out predetermined correction processing with respect to one or more 
items regarding the quality of color images... "at abstract), the method 
comprising: removing a percentage of overlapping pixels with the greatest 
difference in brightness (Refer to Figure 4, numeral #45); generating a first 
histogram of the first region and a second histogram of the second region after 
said removing (Refer to Figure 5); histogram matching the first and the second 
histogram to determine corresponding pixel values from the first and the 
second histograms (Refer to Figure 4, numeral #43-#47); minimizing a color 
matching error between the corresponding pixel values, wherein the color 
matching error is generated from an optoelectronic conversion function (OECF) 
(Refer to Figure 4, numeral #41); and color matching the second image to the 
first image by applying the OECF to the second image (Refer to Figure 4, 
numerals #53-#59). 
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Referring to Claim 17: Masaki discloses the method of claim 16, wherein said 
histogram matching the first and the second histograms comprises generating 
a lookup table (LUT) as follows: (1) initializing all entries in the LUT to 0; 

k 

(2) initializing a first loop by setting i = 0;y « 0; rem I = hl[0]; and rem2 h2[0]; 

(3) updating the LUT by setting minjrem = min(remJ > rem2)\ rem J - rem! - 
minjrem; and rem! >• = rem! -minjrem; and incrementing LUT[i][/] by minjrem\ 

(4) if rem I = 0, then incrementing i and setting rem I = hl[i); 

(5) if rem2 = 0, then incrementings and setting rem2 = h2[/J; 

(6) if / < 256 and y < 256, then repeating steps (3) to (5); 

wherein hl[ ] is the number of pixels having a certain pixel value in the first 
histogram, h2[ ] is the number of pixels having a certain pixel value in the 
second histogram, and LUT[ ][ ] is the number of pairs of corresponding pixel 
values having a certain pixel value in the first histogram and a certain pixel 
value in the second histogram (Refer to Figure 1, numeral 203). 

Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
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skill in the art to which said subject matter pertains. Patentability shall not be negatived by 
the manner in which the invention was made. 

7. Claims 6-7 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Masaki (US 7,215,812 Bl) in combination with Hasler "Modeling the Opto- 

Electronic Conversion Function (OECF) For Application in the Stitching of 

Panoramic Images, hereinafter referred to as Hasler- (MOASPI). 
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Regarding Claim 6: 

Masaki discloses determining at least one parameter of an optoelectronic 
conversion function (OECF) that best matches the corresponding pixel values 
and color matching the second image to the first image by applying the OECF 
with the at least one parameter to the second image. 

Masaki does not specifically disclose wherein said determining at least one 
parameter of an OECF comprises minimizing a color matching error, the color 
matching error being defined as: 



2SS 2SS 

e = J^WTfiJfJJ((i + 1)/ 256.0- $-'(zS((j + 1)1 256.0))) , 

wherein e is 

the color matching error, r is a color matching parameter, and S( ) is the OECF 



Hasler - (MOASPI) teaches the method of claim 5, wherein said determining at 
least one parameter of an OECF comprises minimizing a color matching error, 
the color matching error being defined as: 

2SS 2SS 

e = Y£WT[i)[j]((i + !)/ 256.0 -S->(TS((j + l)n56.0))) , 

wherein e is 

the color matching error, r is a color matching parameter, and S( ) is the OECF 
(Refer to Equation 2b, right column, section 4, "The Error Metric") . 
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At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to define the color matching error as taught by Hasler 
at Equation 2b to be the determination and defining elements of the OECF to 
determine a parameter for application as disclosed by Masaki because this 
function "creates parameters that account for the difference in exposure of the 
two pictures" as disclosed in the claimed invention. 
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Regarding Claim 7: 

Masaki discloses determining at least one parameter of an optoelectronic 
conversion function (OECF) that best matches the corresponding pixel values 
and color matching the second image to the first image by applying the OECF 
with the at least one parameter to the second image. 

Masaki does not specifically disclose wherein the OECF is defined as: 



, wherein x is a pixel value normalized to 
(0,1), and a epsilon.(-l,l) is another color matching parameter (Refer to 
Equation 1, right column, section 3, "The OECF Model"). 

Hasler- (MOASPI) teaches the method of claim 6, wherein the OECF is defined 



(0,1), and a epsilon.(-l,l) is another color matching parameter (Refer to 
Equation 1, right column, section 3, "The OECF Model") . 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to define the OECF as taught by Hasler to be 
incorporated as the function disclosed by Masaki because this function 





, wherein x is a pixel value normalized to 



as: 
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"creates parameters that account for the difference in exposure of the two 
pictures" as disclosed in the claimed invention. 

Regarding Claim 10: 

Masaki discloses determining at least one parameter of an optoelectronic 
conversion function (OECF) that best matches the corresponding pixel values 
and color matching the second image to the first image by applying the OECF 
with the at least one parameter to the second image. 

Masaki does not specifically disclose wherein said applying the optoelectronic 

conversion function comprises: wherein x.sub.o is an 

original pixel value in the second image, x.sub.c is a corrected pixel value in 
the second image, S.sup.-1( ) is the inverse of the OECF, and W is a weight 

W(t f X l )~T+(I-T)x r 

function defined as: 

Hasler- (MOASPI) teaches the method of claim 6, wherein said applying the 

x t = S^(W(t 9 xjS( xj), 
optoelectronic conversion function comprises: wherein 

x.sub.o is an original pixel value in the second image, x.sub.c is a corrected 

pixel value in the second image, S.sup.-1( ) is the inverse of the OECF, and W 
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is a weight function defined as: 



(Refer to Equation 4, left 



column, section 5, "The Complete Model"). 



At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to apply the inverse of the OECF as taught by Hasler to 
the application of the OECF as disclosed by Masaki because "it will avoid the 
degenerate solution and is convenient to approximate." (Hasler, page 2, 
paragraph 3). 

Regarding Claim 11: 

Masaki discloses determining at least one parameter of an optoelectronic 
conversion function (OECF) that best matches the corresponding pixel values. 

Masaki does not specifically disclose the OECF is defined as: 



wherein S( ) is the OECF, x is a pixel value normalized to (0,1), and a.epsilon.(- 
1,1) is a first color matching parameter 

Hasler- (MOASPI) teaches the method of claim 1, wherein the OECF is defined 
as: 
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S( X ) = x + lareJ ; 5in(m > X 
z \l-acos(m)) 

wherein S( ) is the OECF, x is a pixel value normalized to (0,1), and a.epsilon.(- 
1,1) is a first color matching parameter (Refer to Equation 1, right column, 
section 3, "The OECF Model"). 

At the time the invention was made, it would have been obvious to define 
the OECF function as taught by Hasler to be further defined and incorporated 
as the OECF as disclosed by Masaki because the OECF with these definitive 
elements can "deliver an optimal result in a least square error sense. " (Hasler, 
abstract) 

Regarding Claim 12: 

Masaki discloses determining at least one parameter of an optoelectronic 
conversion function (OECF) that best matches the corresponding pixel values. 

Masaki does not specifically disclose wherein said determining at least one 
parameter of 

an OECF comprises minimizing a color matching error defined as: 
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wherein e is the color matching error, x.sub. 1 and x.sub.2 are corresponding 
pixel values in the first and the second regions, R.sub.l and R.sub.2 are the 
first and the second regions, S( ) is the OECF, S( ).sup.-l is the inverse OECF, 
and .tau. is a second color matching parameter. 

Hasler- (MOASPI) teaches the method of claim 11, wherein said determining at 
least one parameter of 

an OECF comprises minimizing a color matching error defined as: 

wherein e is the color matching error, x.sub. 1 and x.sub.2 are corresponding 
pixel values in the first and the second regions, R.sub.l and R.sub.2 are the 
first and the second regions, S( ) is the OECF, S( ).sup.-l is the inverse OECF, 
and .tau. is a second color matching parameter (Refer to Equation 2d, left 
column, paragraph 2, section 4 "The Error Metric"). . 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to add a color matching error based on at least one 
parameter as taught by Hasler to the OECF function as disclosed by Masaki 
because this color matching error as "defined applies to the region of the 
histogram that contains most of the data still delivering a small error." (Hasler, 
page 2, right column, 2 nd paragraph). 
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Regarding Claim 13: 

Masaki discloses determining at least one parameter of an optoelectronic " 
conversion function (OECF) that best matches the corresponding pixel values. 

Masaki does not specifically disclose minimizing a color matching error 
comprises performing a golden section search of the color matching error. 

Hasler- (MOASPI) teaches the method of claim 12, wherein said minimizing a 
color matching error comprises performing a golden section search of the color 
matching error (For example, refer to equations (10.1.6) and (10.1.7) in 
reference to the golden mean or golden section search examples). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to utilize a golden section search of the color matching 
error as taught by Hasler to the OECF function and the corresponding objective 
of minimizing the color matching error as disclosed by Masaki because "the 
golden section search guarantees that each new function evaluation will 
bracket minimum to an interval a precise number times the size of the 
preceding interval." (Press, page 399-400, last paragraph, final sentence). 

Regarding Claim 15: 

Masaki discloses determining at least one parameter of an optoelectronic 
conversion function (OECF) that best matches the corresponding pixel values. 
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Masaki does not specifically disclose applying the OECF comprises: 

wherein x.sub.o is an original 
pixel value in the second image, x.sub.c is a corrected pixel value in the 

W(t 9 X t )*T + (I-T)x r 

second image, and W is a weight function defined as: 

Hasler- (MOASPI) teaches the method of claim 12, wherein said applying the 

OECF comprises: wherein x.sub.o is an original 

pixel value in the second image, x.sub.c is a corrected pixel value in the 

W(t % x i )^rHi-T)x r 

second image, and W is a weight function defined as: 

(Refer to Equation 4, left column, section 5, "The Complete Model?). 

At the time the invention was made, it would have been obvious to one of 

x t =Sr*(W(x.xJS(x.)) t 
ordinary skill in the art to apply the OECF as and define 

W(t >X))**T+(l-T)X r 

the weight function as as taught by Hasler and to 

further define the OECF function as disclosed by Masaki because with the 
application of Equation 4 (Hasler), applies the gains to adjust the white points 
and reapply the OECF." (Hasler, page 2, section 5, paragraph 2). 
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Regarding Claim 18: 

Masaki discloses minimizing a color matching error between the corresponding 
pixel values, wherein the color matching error is generated from an 
optoelectronic conversion function (OECF) (Refer to Figure 4, numeral #41). 

Masaki does not specifically disclose determining parameters of the OECF 
comprises minimizing a color matching error, the color matching error being 



defined as: 

wherein e is the color matching error, .tau. is a color matching parameter 
(Refer to Equation 2b, right column, section 4, "The Error Metric"); and S(x) is 



x is a pixel value normalized to (0,1), and a.epsilon.(-l,l) is another color 
matching parameter. 

Hasler- (MOASPI) teaches the method of claim 17, wherein said determining 
parameters of the OECF comprises minimizing a color matching error, the color 
matching error being defined as: 



e 



255 255 

= YL L UT[i}[jJ((^l)/256.0-S-'(tS((j + J)/ 256.0))) , 



the OECF defined as: 




wherein S(x) is the OECF, 



2SS 2SS 

e = ^?Wr[i][j]((i + l)/256.0-S'(TS((j + l)/256.0))), 
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wherein e is the color matching error, .tau. is a color matching parameter 
(Refer to Equation 2b, right column, section 4, "The Error Metric"); and S(x) is 



x is a pixel value normalized to (0,1), and a.epsilon.(-l,l) is another color 
matching parameter (Refer to Equation 1, right column, section 3, "The OECF 
Model"). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to further define the OECF function as disclosed by 
Masaki, with reference to Equation 1, right column, section 3, "The OECF 
Model" as taught by Hasler because "the OECF with these definitive elements 
can "deliver an optimal result in a least square error sense." (Hasler, abstract). 
Regarding Claim 20: 

Masaki discloses color matching the second image to the first image by 
applying the OECF to the second image. 

Masaki does not specifically disclose applying the optoelectronic conversion 
function comprising: 



image, x.sub.c is a corrected pixel value of the second image, and W is a weight 
function defined 



the OECF defined as: 




wherein S(x) is the OECF, 



wherein x.sub.o is an original pixel value in the second 
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W(t,X t )=t + (I-T)x r 

as: 



Hasler- (MOASPI) teaches the method of claim 19, wherein said applying the 
optoelectronic conversion function comprises: 

wherein x.sub.o is an original pixel value in the second 
image, x.sub.c is a corrected pixel value of the second image, and W is a weight 
function defined 

as: (Refer to Equation 4, left column, section 5, "The 

Complete Model"). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to further define the OECF function as disclosed by 
Masaki to determine additional application with reference to corrected pixel 
values as taught by Hasler at Equation 4 because u this application performs 
the balancing by using the weight factor and adjusts the white point and 
reapplies the OECF." (Hasler, Page 2, section 5, paragraph 2). 
8. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Masaki (US 7,215,812 Bl) in combination with Hasler, "Modeling the Opto- 
Electronic Conversion Function (OECF) For Application in the Stitching of 
Panoramic Images, as applied to claims 1-7 above, and further in view of Press, 
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"The Art of Scientific Computing, 10.1 Golden Search in One Dimension"- 
hereinafter referred to as Press. 
Regarding Claim 8: 

Masaki and Hasler disclose determining at least one parameter of an 
optoelectronic conversion function (OECF) that best matches the corresponding 
pixel values and color matching the second image to the first image by applying 
the OECF with the at least one parameter to the second image wherein the 

x \l-acos(fDc)j 

OECF is defined as: , wherein x is a pixel 

value normalized to (0,1), and a epsilon.(-l,l) is another color matching 
parameter (Refer to Equation 1, right column, section 3, "The OECF Model"). 

The combination of Masaki and Hasler does not specifically disclose wherein 
said minimizing a color matching error comprises performing a golden section 
search of the color matching error (For example, refer to equations (10.1.6) and 
(10.1.7) in reference to the golden mean or golden section search examples). 

Press teaches the method of claim 7, wherein said minimizing a color matching 
error comprises performing a golden section search of the color matching error 
(For example, refer to equations (10.1.6) and (10.1.7) in reference to the golden 
mean or golden section search examples). 
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At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to perform a golden section search of the color 
matching error as illustrated through examples from Press and combine that 
search together with the "OECF" as disclosed by the combined teaching of 
Masaki and Hasler because "the golden section search guarantees that each 
new function evaluation will bracket minimum values to an interval at a 
precise number times the size of the preceding interval." (Press, page 399-400, 
last paragraph, final sentence). 

Allowable Subject Matter 

9. Claims 9,14 and 19 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including 
all of the limitations of the base claim and any intervening claims. 



Conclusion 

10. The prior art made of record and not relied upon is considered pertinent 
to applicant's disclosure. 

2003/0103677 5,664,027 
2007/0 172 12 1 6,798,92 1 

6,859,552 6,731,792 
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• Szeliski et al. "Creating Full View Panoramic Image Mosaics and 
Environment Maps". 1997, pages 251-258. 

Any inquiry concerning this communication or earlier communications 
from the examiner should be directed to Mia M. Thomas whose telephone 
number is 571-270-1583. The examiner can normally be reached on Monday- 
Friday 8:30am-5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Brian Werner can be reached on 571-272-7401. The 
fax phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see 
http://pair-direct.uspto.gov. Should you have questions on access to the 
Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 
9197 (toll-free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 
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Mia M Thomas 
Examiner 
Art Unit 2624 



/Brian P. Werner/ 

Supervisory Patent Examiner (SPE), Art Unit 2624 



